Apparatus, method, and computer-readable storage medium for contextualized equipment recommendation

ABSTRACT

The present disclosure relates to a method for providing a user with a contextualized evaluation of a fit of frames of eyeglasses to their face. In particular, the present disclosure relates to a method, comprising receiving user data describing features of the face of the user, receiving equipment data describing features of the eyeglass frame, generating, according to a first model, values for a set of specific criteria describing compatibility between the face of the user and the eyeglass frame, the first model trained to associate user data and equipment data with values of specific criteria, generating, according to a second model, a value of a global criterion based on the generated values for the set of specific criteria, the second model trained to associate the values of specific criteria with values of global criteria, determining a message characterizing the eyeglass frame with respect to the face of the user.

BACKGROUND FIELD OF THE DISCLOSURE

The present disclosure relates to eyewear and, specifically, to matchingof visual equipment with user faces.

DESCRIPTION OF THE RELATED ART

During the selection of new visual equipment, or eyewear, a user isoften left to self-reflection in determining the aesthetics of neweyewear on their face. Moreover, when deciding between multiple piecesof eyewear, a user may find it difficult to decide which piece is mostattractive, has the most utility, or is the most suited to theirparticular facial bone structure and features. At the same time, thepatient may be grappling with their own opinion of the new eyewear ontheir face and the hypothetical opinions of third parties (e.g.,friends, family, professionals, etc.) regarding the fit of the neweyewear on their face.

As demonstrated above, considering the aesthetic appeal together withthe eyewear necessity of proper vision, the task of eyewear selectioncan be burdensome, with no effective way of confidently purchasing a newset of eyewear that the user, the user's doctor, and the user's friendsare sure to be pleased with. The present disclosure provides a solutionto this issue.

The foregoing “Background” description is for the purpose of generallypresenting the context of the disclosure. Work of the inventors, to theextent it is described in this background section, as well as aspects ofthe description which may not otherwise qualify as prior art at the timeof filing, are neither expressly or impliedly admitted as prior artagainst the present invention.

SUMMARY

The present disclosure relates to an apparatus, method andcomputer-readable storage medium for contextualized equipmentrecommendation.

According to an embodiment, the present disclosure is further related toa method for providing contextual evaluation of an eyeglass frame on aface of a user, comprising receiving user data describing features ofthe face of the user, receiving equipment data describing features ofthe eyeglass frame, generating, according to a first model, values for aset of specific criteria describing compatibility between the face ofthe user and the eyeglass frame based on the received user data and thereceived equipment data, the first model trained to associate user dataand equipment data with values of specific criteria, generating, byprocessing circuitry and according to a second model, a value of aglobal criterion based on the generated values for the set of specificcriteria, the second model trained to associate the values of specificcriteria with values of global criteria, determining a messagecharacterizing the eyeglass frame with respect to the face of the user,the message being associated with the generated value of the globalcriterion and with the generated values for the set of specificcriteria, and outputting the message to the user.

According to an embodiment, the present disclosure is further related toan apparatus for providing contextual evaluation of an eyeglass frame ona face of a user, comprising processing circuitry configured to receiveuser data describing features of the face of the user, receive equipmentdata describing features of the eyeglass frame, determine, according toa first model, values for a set of specific criteria describingcompatibility between the face of the user and the eyeglass frame basedon the received user data and the received equipment data, the firstmodel trained to associate user data and equipment data with values ofspecific criteria, generate, according to a second model, a value of aglobal criterion based on the generated values for the set of specificcriteria, the second model trained to associate the values of specificcriteria with values of global criteria, determine a messagecharacterizing the eyeglass frame with respect to the face of the user,the message being associated with the generated value of the globalcriterion and with the generated values for the set of specificcriteria, and output the message to the user.

According to an embodiment, the present disclosure is further related toa non-transitory computer-readable storage medium storingcomputer-readable instructions that, when executed by a computer, causethe computer to perform a method for providing contextual evaluation ofan eyeglass frame on a face of a user, comprising receiving user datadescribing features of the face of the user, receiving equipment datadescribing features of the eyeglass frame, generating, according to afirst model, values for a set of specific criteria describingcompatibility between the face of the user and the eyeglass frame basedon the received user data and the received equipment data, the firstmodel trained to associate user data and equipment data with values ofspecific criteria, generating, according to a second model, a value of aglobal criterion based on the generated values for the set of specificcriteria, the second model trained to associate the values of specificcriteria with values of global criteria, determining a messagecharacterizing the eyeglass frame with respect to the face of the user,the message being associated with the generated value of the globalcriterion and with the generated values for the set of specificcriteria, and outputting the message to the user.

The foregoing paragraphs have been provided by way of generalintroduction, and are not intended to limit the scope of the followingclaims. The described embodiments, together with further advantages,will be best understood by reference to the following detaileddescription taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete appreciation of the disclosure and many of the attendantadvantages thereof will be readily obtained as the same becomes betterunderstood by reference to the following detailed description whenconsidered in connection with the accompanying drawings, wherein:

FIG. 1 is an illustration of a user wearing an eyeglass frame, accordingto an exemplary embodiment of the present disclosure;

FIG. 2 is a flow diagram of a method for providing contextual evaluationof an eyeglass frame, according to an exemplary embodiment of thepresent disclosure;

FIG. 3A is a flow diagram of a method for providing contextualevaluation of an eyeglass frame, according to an exemplary embodiment ofthe present disclosure;

FIG. 3B is a flow diagram of a method for providing contextualevaluation of an eyeglass frame, according to an exemplary embodiment ofthe present disclosure;

FIG. 4A is an illustration of an image of a face of a user, according toan exemplary embodiment of the present disclosure;

FIG. 4B is an illustration of an image of an eyeglass frame, accordingto an exemplary embodiment of the present disclosure;

FIG. 4C is an illustration of an image of a user wearing an eyeglassframe, according to an exemplary embodiment of the present disclosure;

FIG. 5 is a schematic of a database including user features, equipmentfeatures, and corresponding images of users wearing equipment, accordingto an exemplary embodiment of the present disclosure;

FIG. 6A is a flow diagram of an aspect of a method for providingcontextual evaluation of an eyeglass frame, according to an exemplaryembodiment of the present disclosure;

FIG. 6B is an illustration of a survey administered to eye careprofessionals, according to an exemplary embodiment of the presentdisclosure;

FIG. 7A is a flow diagram of an aspect of a method for providingcontextual evaluation of an eyeglass frame, according to an exemplaryembodiment of the present disclosure;

FIG. 7B is an illustration of metric used in determining a specificcriterion, according to an exemplary embodiment of the presentdisclosure;

FIG. 7C is an illustration of metric used in determining a specificcriterion, according to an exemplary embodiment of the presentdisclosure;

FIG. 7D is an illustration of metric used in determining a specificcriterion, according to an exemplary embodiment of the presentdisclosure;

FIG. 7E is an illustration of metric used in determining a specificcriterion, according to an exemplary embodiment of the presentdisclosure;

FIG. 7F is an illustration of metric used in determining a specificcriterion, according to an exemplary embodiment of the presentdisclosure;

FIG. 7G is an illustration of metric used in determining a specificcriterion, according to an exemplary embodiment of the presentdisclosure;

FIG. 7H is an illustration of metric used in determining a specificcriterion, according to an exemplary embodiment of the presentdisclosure;

FIG. 7I is an illustration of metric used in determining a specificcriterion, according to an exemplary embodiment of the presentdisclosure;

FIG. 7J is an illustration of metric used in determining a specificcriterion, according to an exemplary embodiment of the presentdisclosure;

FIG. 7K is an illustration of metric used in determining a specificcriterion, according to an exemplary embodiment of the presentdisclosure;

FIG. 7L is an illustration of metric used in determining a specificcriterion, according to an exemplary embodiment of the presentdisclosure;

FIG. 8A is a graphical representation of responses to a surveyadministered to eye care professionals, according to an exemplaryembodiment of the present disclosure;

FIG. 8B is a graphical representation of estimated responses to a surveyadministered to eye care professionals, according to an exemplaryembodiment of the present disclosure;

FIG. 9A is a flow diagram of an aspect of a method for providingcontextual evaluation of an eyeglass frame, according to an embodimentof the present disclosure;

FIG. 9B is a flow diagram of a decision tree of an aspect of a methodfor providing contextual evaluation of an eyeglass frame, according toan embodiment of the present disclosure;

FIG. 10A is a flow diagram of an aspect of a method for providingcontextual evaluation of an eyeglass frame, according to an embodimentof the present disclosure;

FIG. 10B is a flow diagram of an annotated decision tree of an aspect ofa method for providing contextual evaluation of an eyeglass frame,according to an embodiment of the present disclosure;

FIG. 11 is hardware configuration of a frame fit evaluation device,according to an exemplary embodiment of the present disclosure; and

FIG. 12 is a flow diagram of an aspect of a method for providing acontextualized evaluation of an equipment data when other equipment dataare provided.

DETAILED DESCRIPTION

The terms “a” or “an”, as used herein, are defined as one or more thanone. The term “plurality”, as used herein, is defined as two or morethan two. The term “another”, as used herein, is defined as at least asecond or more. The terms “including” and/or “having”, as used herein,are defined as comprising (i.e., open language). The terms “visualequipment”, “equipment”, “equipments”, “eyeglass frame”, “eyeglassframes”, “eyeglass”, “eyeglasses”, and “visual equipments” may be usedinterchangeably to refer to an apparatus having both a frame and a lens.The term “visual equipment” may be used to refer to a single visualequipment while the term “visual equipments” may be used to refer tomore than one visual equipment.

Reference throughout this document to “one embodiment”, “certainembodiments”, “an embodiment”, “an implementation”, “an example” orsimilar terms means that a particular feature, structure, orcharacteristic described in connection with the embodiment is includedin at least one embodiment of the present disclosure. Similarly, theterms “image of a face” and “image of a face of a person” arecorresponding terms that may be used interchangeably. Thus, theappearances of such phrases or in various places throughout thisspecification are not necessarily all referring to the same embodiment.Furthermore, the particular features, structures, or characteristics maybe combined in any suitable manner in one or more embodiments withoutlimitation.

Today, patients, users, or consumers in search of eyeglasses are oftenleft with little guidance as to what is both ophthalmologicallyappropriate and aesthetically pleasing. For some, cultural trends drivetheir decision making. For others, the opinion of friends and family ismost important. For others still, who prioritize an ergonomic fit andvisual acuity, the opinion of a trained eye care professional (ECP) is anecessity.

Currently, users have access to approaches that provide some but not allof the above-described features. For instance, one approach describesimplementation of a decision tree to match eyeglass frames withmorphological features detected from landmarks on the face of anindividual, the match determining a pair of eyeglasses that best matchesthe individual. In another approach, a user questionnaire may be used tomatch style preferences of the user to available eyeglass frames. Ineither scenario, and as is the case generally, these approaches providea user with knowledge that a particular pair of eyeglasses does or doesnot it. These approaches do not, however, provide context to thedetermination. For example, while these approaches may be able torecommend an eyeglass frame to a user based on style preferences in viewof best-selling rankings and the like, the recommendation ultimatelyreflects a single ‘best fit’ metric. The single ‘best fit’ metric, whilebased on underlying features of the eyeglass frames and the user,provides an oversimplification of the ‘fit’ of the eyeglass frame andfails to convey to the user why the recommended eyeglass frame is the‘best fit’. In certain cases, the oversimplification may be aquantitative metric (e.g. between 1 and 10) based on a global mark ofthe eyeglass frame or specific criteria, leaving interpretation of themetric to an ECP. In this way, while providing a user with the knowledgethat a particular pair of eyeglasses does or does not fit, theseapproaches fail to provide textual context to a user regarding why sucha determination was made (e.g., why the frame fits or does fit accordingto specific features thereof).

U.S. Patent Application Publication No. 2017/169501 describes a databaseincluding eyeglass models, user face models, and an eyeglass fitevaluation model based on a fitting evaluation matrix. While providing afit output, the eyeglass fit evaluation model only generates a single,global metric in determining the fit of a certain eyeglass to a face ofa user.

According to an embodiment, the present disclosure describes anapparatus, a method, and a computer-readable storage medium forproviding contextual evaluation of an eyeglass frame on a face of auser.

In an embodiment, the present disclosure provides for the association ofdifferent textual descriptions for each of a subset of possible valuestaken by criterion linked to a global or specific attribute ofsuitability of an eyeglass relative to a user face.

In an embodiment, the present disclosure includes an automaticdiagnostic system for determining a fit between an eyeglass and a user.The automatic diagnostic system may generate at least one fit metricvalue and an associated textual description explaining the reasons whythe eyeglass does or does not suit the face of the user. In an example,the at least one fit metric and the associated textual description canbe based on one or more photos of the user, user information includingeyeglass prescription, age, gender and the like, and equipment featuresincluding size, color, materials, and the like.

In an embodiment, the present disclosure concerns an apparatus, method,and computer-readable storage medium for providing the determination ofuser data describing features of the face of the user and thedetermination of equipment data describing features of the eyeglassframe from at least one picture of the said user wearing the saideyeglass. Digital image processing and other image processing methodscan be used to separate user features from frame features. The user datamay be morphological features, structural features, and aestheticfeatures of the face of the user. The equipment data may be featuresincluding total width of the frame of the equipment, dimensions ofaspects of the frame of the equipment (e.g., Size A, Size B, Size D,etc.), vertical thickness of the top part of the frame of the equipment,horizontal thickness of the frame of the equipment at the level of thehinge, color of the equipment, material of the equipment, and the like.The at least one picture of the face of the user can be produced with a2D or 3D camera, or other image capture device configured to acquireimages of users, eyewear, and the like. At least one fit metric valueand an associated textual description explaining the reasons why theeyeglass does or does not suit the face of the user can be generatedfrom the determined user data and equipment data.

According to an exemplary embodiment, the present disclosure describes amachine learning-based frame fit evaluation device (i.e., eyeglassevaluation tool) for presenting a user with a fitted eyeglass selectionbased upon morphological and structural features (i.e., user featuresand equipment features), ophthalmological demands (i.e. visualprescriptions), and aesthetic appeal.

According to an exemplary embodiment, the present disclosure includes amethod for equipment recommendation based on: (1) user features,equipment features, and positional data of worn equipment, (2) a set ofcriteria values, specific and global, related to fit of the equipment,(3) global criterion based on a global score, grade, or acceptance offit of the equipment, and (4) specific criteria based on a score, grade,or acceptance of specific fit parameters, wherein the global criterioncan be derived from the specific criteria.

As introduced above, the present disclosure provides eyeglassrecommendations and evaluations with textual context, allowing a user tounderstand why equipment does or does not fit. The evaluations can bebased on criteria values, both global and specific, that are determinedfrom models generated by machine learning-based approaches. Each of themachine learning-based approaches can include datasets annotated byECPs, the annotations providing labels for each of the specific criteriaand the global criterion for a corresponding pair of user features andequipment features.

In an embodiment, the present disclosure may provide a global criterionas modified by only relevant specific criteria.

In an embodiment, the present disclosure may provide a global criterion,a specific criteria, or textual information directly to a user.

In an embodiment, the first and second models can be updated based onECP recommendations. The ECP associate a frame model—or its geometricalfeatures—to a picture of the face of a user and declares the score to acentral server via a network. As can be appreciated, the network can bea public network, such as the Internet, or a private network such as anLAN or WAN network, or any combination thereof and can also include PSTNor ISDN sub-networks. The network can also be wired, such as an Ethernetnetwork, or can be wireless such as a cellular network including EDGE,3G, 4G and 5G wireless cellular systems. The wireless network can alsobe WiFi, Bluetooth, or any other wireless form of communication that isknown. The models are updated in real time and other ECPs benefit fromthe update when determining a fit between an eyeglass and a user.

In an embodiment, global criterion can be derived from values ofspecific criteria using machine learning-based approaches. The machinelearning-based approaches can include classification and regressiontrees and/or linear or non-linear combinations of models of specificcriteria.

According to an embodiment, the present disclosure provides multiplecriteria ECP input and machine learning-based multiple related models.These machine learning-based models are not limited to only globalcriterion, instead providing textual information regarding specificcriteria deemed most relevant to the overall ‘fit’ of a frame, orequipment. For instance, it may be determined that equipment does notfit a user well and that the primary driver for this ill-fit is thethickness of the frame of the equipment. Therefore, the user may searchfor similarly related equipment with reduced thickness frames in orderto improve their overall ‘fit’.

According to an embodiment, the present disclosure relates to anapparatus, method, and computer-readable storage medium for providing acontextualized evaluation of an equipment data when other equipment dataare provided. Such a method is illustrated on FIG. 12 . When presentedwith a first set of equipment data where an equipment data is missing,methods of the present disclosure compare the set of data to set of dataof a database comprising at least one set of equipment data. The meanvalue of the missing data from frames of the database having similarother features is then calculated. For instance, the mean thickness ofthe frame can be calculated knowing that it is a frame for women inplastic. The database can provide suggestions of equipments having amissing data value equal or close to the calculated mean value of themissing data and having other values equal or close to the other valuesof the first set of equipment data. The tolerance value on the missingdata value or on one or several other data can be individually selectedfor each data by the user.

According to an embodiment, the present disclosure describes anapparatus, method, and computer-readable storage medium for providing acontextualized evaluation of a fit of equipment and a face of a user.When presented with an image of a user wearing equipment, methods of thepresent disclosure process the image such that a global criterion (i.e.universal fit metric) can be provided with context of features thatcontribute to the global criterion (e.g. specific criteria). In otherwords, the methods herein may determine the equipment and the face ofthe user to be a good match, however, the recommendation may bequalified by stating that the equipment and the face of the user are agood match because the relative distance between the centers of thelenses of the equipment and the interpupillary distance is aestheticallyappropriate. Or, in an embodiment, the recommendation may be qualifiedby stating the equipment and the face of the user are a good matchbecause the distance between the centers of the lenses is smaller thanthe interpupillary distance, the pupils of the user then beingpositioned closer to a nasal component of the lenses of the equipment.

According to an embodiment, the above-described multiple criteria ECPinput and machine learning-based multiple related models provide forincreasingly robust and accurate results for contextualized globalcriterion.

Turning now to the Figures, FIG. 1 is an illustration of a user wearingan eyeglass frame, according to an exemplary embodiment of the presentdisclosure. The illustration of

FIG. 1 can be an input to the method of the present disclosure, in anexample. Features of the user and features of the equipment can bedetermined therefrom. In this spirit, it can be appreciated that aninput to the method of the present disclosure may, alternatively or inaddition, be an image of a face of a user, an image of equipment, oruser features data and equipment features data previously determinedfrom like images.

Inputs similar to FIG. 1 may be applied to a method of the presentdisclosure as in FIG. 2 , which describes an implementation of themethods of the present disclosure.

FIG. 2 is a flow diagram of a method for providing contextual evaluationof an eyeglass frame, according to an exemplary embodiment of thepresent disclosure. It can be appreciated that method 200 can beperformed by a frame fit evaluation device, the frame fit evaluationdevice including processing circuitry configured to perform the stepsdescribed herein. The frame fit evaluation device will be described ingreater detail with reference to FIG. 11 .

At step 210 of method 200, user data may be received. The user data canbe, as discussed with reference to FIG. 3A and FIG. 3B, provideddirectly as user features or can be determined from images containing aface of the user.

At step 220 of method 200, equipment data may be received. The equipmentdata can be, as discussed with reference to FIG. 3A and FIG. 3B,provided directly as equipment features or can be determined from imagescontaining equipment.

At step 230 of method 200, specific criteria values can be generated byapplying the above user features and equipment features to a machinelearning-based model of specific criteria. Values of specific criteriametrics can be based on a set of specific criteria metrics directed tosuitability of equipment and a face of a user, according to particularmorphological, aesthetic, or visual considerations. In an embodiment,the specific criteria can be numeric values and continuous quantities,such as probabilities, combinations of quantities, scores, and the like.For instance, the specific criteria may define an interpupillarydistance of a user. In an embodiment, the specific criteria can bequalitative quantities that may be defined by alphanumeric values. Forinstance, a qualitative quantity may represent an evaluation, by an ECP,of a width of a frame of equipment relative to a width of a face of auser. The ECP may then decide whether the relative width is (a) toowide, (b) acceptable, or (c) too narrow. In another instance, aqualitative quantity may represent an evaluation, by an ECP, of apresence of an eyebrow of a user within frames of equipment. The ECP maythen decide whether (a) the eyebrow is visible inside the frames, (b)eyebrow positioning is acceptable, or (c) the eyebrow is too high abovethe frames. Such examples are merely representative of a variety ofqualitative quantities that are relevant to frame fit and that may beevaluated by an ECP.

At step 240 of method 200, a global criterion value can be generated byapplying the generated specific criteria values to a machinelearning-based model of global criterion. In an embodiment, the globalcriterion value can be further based on the above-defined user featuresand equipment features. The global criterion value can be a numeric orqualitative value indicating a global suitability of equipment to theface of the user. In an embodiment, a machine learning-based model maybe used to generate the global criterion value from the generatedspecific criteria values. The machine learning-based model may be adecision tree generated by a classification and regression tree or maybe a linear regression of the specific criteria values.

According to an embodiment, in each of step 230 and step 240, specificcriteria values and global criterion values can be determined accordingto machine learning-based approaches generated based on input from ECPs.For instance, the machine learning-based approaches may be based on ECPassessment and input of specific criteria values and global criterionfor a given set of images of worn equipment, as will be described later.

At step 250 of method 200, the specific criteria values generated atstep 230 and global criterion value generated at step 240 may beevaluated to determine a message characterizing fit. In this way, theglobal criterion value can be contextualized by pertinent specificcriteria values, thereby providing a comprehensive, text- andlanguage-based output as an alternative to value-based outputs that lackmeaning. In an embodiment, the evaluation of the global criterion andthe specific criteria can be performed by decision tree, whereinspecific criteria values inform and contextualize the global criterionfound at an end of a branch of the decision tree. The global criterion(e.g., “fit”, “no fit”) may be provided alongside pertinent specificcriteria (e.g. “color mismatch”) that form the basis of the globalcriterion. The decision tree may be an annotated decision tree andinclude bifurcations defining varying semantic text templates, eachbifurcation or path being defined by evaluations in specific criteriavalues and resulting in a global criterion value. Each resulting globalcriterion value can then be described using lay terminologycontextualized by the features of the specific criteria, and informed byan ECP, that defined the path thereto (as described in step 260). In anembodiment, the resulting textual description can include textualtranslations of the reasons of bifurcation. An annotated decision treedescribed above will be further described with reference to FIG. 10A andFIG. 10B.

At step 260 of method 200, the message determined at step 250 may beoutput to the user. In an embodiment, the determined message may bedirectly provided as an output to the user. In another embodiment, thedetermined message may be modified according to automatic naturallanguage generation tools to produce a more naturalized message inaccordance with preferences and habits of the frame fit evaluationdevice. For instance, the modification may result in a given messagebeing provided in multiple ways so as not to appear redundant anddisingenuous when the same bifurcations of the tree are followed fordifferent equipment. In an example, the result may be a rephrasing ofthe textual message or a different contextualization of the globalcriterion value by the specific criteria values. Instead of defining theglobal criterion value by dimensions of the equipment relative to theface of the user, the global criterion value may be defined accordingto, as a specific criteria value, a color of the equipment relative to askin color of the user. In an embodiment, the automatic natural languagegeneration tools may consider impact of certain specific criteria on theglobal criterion value such that, when providing alternative humanoidexplanations for a global criterion value, the textual context may bemeaningful. For instance, a color of equipment and dimensions of theequipment may be specific criteria with equal impact on the globalcriterion value and, therefore, may be used interchangeably by theautomatic natural language generation tools.

In an embodiment, the message can be output to the user by a variety ofmodalities including audio, video, haptic, and the like. In an example,the message can be delivered to the user by spoken word.

Focusing now on each step of method 200, step 210 and step 220 will bedescribed in further detail with respect to FIG. 3A and FIG. 3B. FIG. 3Aand FIG. 3B are each flow diagrams of a method for providingcontextualized evaluation of an eyeglass frame, according to exemplaryembodiments of the present disclosure.

Regarding FIG. 3A, step 210 of method 200 and step 220 of method 200 areperformed concurrently and each are performed on separate images and/ordata. For instance, a user may provide an image of their face and wishto request a contextualized evaluation of a hypothetical match betweenthe image and selected equipment from an equipment database.

Accordingly, at step 311 of method 201, user features may be extractedfrom, at step 312, a user image of a face of the user or from userfeatures stored within a user features database. The user featuresdatabase may include a user profile associated with the user, whereinuser features have been input by the user for storage and access. Theuser features may be morphological features, structural features, visualprescriptions, and aesthetic features of the face of the user. At step321 of method 201, equipment features may be extracted from, at step322, an equipment image of equipment or from equipment features storedwithin an equipment features database. The equipment features databasemay include equipment features associated with a plurality of equipmentstored in an online inventory, each set of equipment features definingstructural features, aesthetic features, and visual features of theequipment. Alternatively, as suggested, the user features and theequipment features can be extracted from respective images at step 311and step 321.

Similar to method 200 of FIG. 2 , the user features and the equipmentfeatures extracted at step 311 and step 321 can be applied to a model ofspecific criteria at step 331 of method 201. With application of themodel on the user features and the equipment features, values ofspecific criteria metrics can be generated according to a set ofspecific criteria metrics directed to suitability of equipment relativeto a face of a user. The suitability of equipment may be definedaccording to particular morphological, aesthetic, or visualconsiderations. In an embodiment, the specific criteria can be numericvalues and continuous quantities, such as probabilities, combinations ofquantities, scores, and the like. In an embodiment, the specificcriteria can be qualitative quantities that may be defined byalphanumeric values.

Similar to method 200 of FIG. 2 , the values of specific criteriadetermine at step 331 of method 201 can be applied to a model of globalcriterion at step 341 of method 201. A resulting global criterion valuecan be a numeric or qualitative value indicating a global suitability ofequipment to the face of the user. In an embodiment, a machinelearning-based model may be used to generate the global criterion valuefrom the generated specific criteria values. The machine learning-basedmodel may be a decision tree generated by a classification andregression tree or may be a linear regression of the specific criteriavalues.

Having generated the specific criteria values and the global criterionvalue via, in an example, an annotated decision tree, method 201 canreturn to method 200 of FIG. 2 wherein a message can be determined andoutput to a user at steps 250 and 260, respectively.

Regarding FIG. 3B, step 210 of method 200 and step 220 of method 200 areperformed concurrently and each are performed on images and/or datareflecting an image of a face of a user wearing equipment. For instance,a user may provide an image of their face wearing equipment and may wishto request a contextualized evaluation of the ‘fit’ therebetween. It canbe appreciated that such an instance may be relevant when a user is at aretail store and is trying on a variety of equipment, or when the useris online shopping and is virtually ‘trying on’ a variety of equipment.

Accordingly, at step 311 of method 201, user features may be extractedfrom, at step 302, a user image of a face of the user wearing equipment.The user features may be morphological features, structural features,and aesthetic features of the face of the user. At step 321 of method201, equipment features may be extracted from, at step 302, the userimage of the face of the user wearing the equipment. The equipmentfeatures may define structural features, aesthetic features, and visualfeatures of the equipment.

Similar to method 200 of FIG. 2 , the user features and the equipmentfeatures extracted at step 311 and step 321 can be applied to a model ofspecific criteria at step 331 of method 201. With application of themodel to the user features and the equipment features, values ofspecific criteria metrics can be generated according to a set ofspecific criteria metrics directed to suitability of equipment and aface of a user. The suitability of equipment may be defined according toparticular morphological, aesthetic, or visual considerations. In anembodiment, the specific criteria can be numeric values and continuousquantities, such as probabilities, combinations of quantities, scores,and the like. In an embodiment, the specific criteria can be qualitativequantities that may be defined by alphanumeric values.

Similar to method 200 of FIG. 2 , the values of specific criteriadetermine at step 331 of method 201 can be applied to a model of globalcriterion at step 341 of method 201. A resulting global criterion valuecan be a numeric or qualitative value indicating a global suitability ofequipment to the face of the user. In an embodiment, a machinelearning-based model may be used to generate the global criterion valuefrom the generated specific criteria values. The machine learning-basedmodel may be a decision tree generated by a classification andregression tree or may be a linear regression of the specific criteriavalues.

Having generated the specific criteria values and the global criterionvalue via, in an example, an annotated decision tree, method 201 canreturn to method 200 of FIG. 2 wherein a message can be determined andoutput to a user at steps 250 and 260, respectively.

In view of the above, FIG. 4A, FIG. 4B, and FIG. 4C represent possibleinputs to method 200 of FIG. 2 . In an example, a user may provide auser image 412 independent of equipment. In this way, the user mayselect any one of a number of images of equipment 422 that may beevaluated to provide a contextualized evaluation of the face of the userand the equipment. Alternatively, or as a virtual ‘try on’, the user mayprovide a user image 412 and equipment image 422 as a combined image 402of the user wearing the equipment. In any of the cases presented above,the images may be processed in order to extract user features andequipment features.

The above descriptions have focused on a flow diagram as experienced byan end user. In an exemplary embodiment of the present disclosure, anend user may provide an image of their face wearing equipment. To allowfor this operation, the specific criteria model and the global criterionmodel must be developed.

With reference to FIG. 5 , the specific criteria model can be initiallydeveloped by generating a database of user features, equipment features,and corresponding images of users wearing equipment. To this end, rawdatabase A 533 may include a plurality of datasets comprising userfeatures 511, equipment features 521, and corresponding images of theuser wearing the equipment 502. The corresponding images of the userwearing the equipment 502 may be one image or a plurality of images. Inan embodiment, the plurality of datasets of raw database A 533 may be anumber of datasets sufficient to generate an accurate model of specificcriteria while minimizing computational burdens.

According to an exemplary embodiment, the user features 511 may beextracted from the corresponding image of the user wearing the equipment502, from an image of the user without the equipment, from morphologicaldata associated with the user, and the like. The images may be acquiredusing a two-dimensional imaging device or a three-dimensional imagingdevice.

According to an exemplary embodiment, the equipment features 521 may beextracted from the corresponding image of the user wearing the equipment502, from an image of the equipment without the face of the user, fromstructural data associated with the equipment according to athree-dimensional rendering of the design of the equipment, fromstructural data acquired by measurement using a frame trace device, oranother source. The images may be acquired using a two-dimensionalimaging device or a three-dimensional imaging device.

According to an exemplary embodiment, the image of the user wearing theequipment 502 can be an image of a real user wearing the equipment, animage of the real user wearing a virtual ‘try on’ of equipment, an imageof a virtual user (i.e., an avatar) wearing a virtual ‘try on’ ofequipment, and the like. User features 511 and equipment features 521can be acquired therefrom according to the techniques described above.

Returning to method 200, the specific criteria model and the globalcriterion model can be developed based on the user features and theequipment features in consultation with ECPs. The specific criteriamodel and the global criterion model can be based on evaluations by ECPsof images of faces of users wearing equipment. By completing an ECPsurvey, as shown in FIG. 6B, the ECP can evaluate the image of the faceof the user wearing the equipment according to a set of specificcriteria related to aspects of the ‘combined’ image and according to aglobal criterion.

As an overview to the ECP evaluation process, and with reference to FIG.6A, raw dataset A 633, including images of faces of users wearingequipment 602, may be evaluated by a plurality of ECPs 634 according toan ECP survey. Shown in FIG. 6B, the ECP survey may include specificcriteria survey questions and a global criterion survey question. Forinstance, the survey may include, as specific criteria the following:

Specific Criteria 1: Provide feedback regarding the width of theequipment relative to the size of the head of the user. A high negativescore indicates the width of the equipment is too small, while a highpositive score indicates the width of the equipment is too large.

Specific Criteria 2: Provide feedback regarding the location of a pupilrelative to a shape of the equipment/lens. A high negative scoreindicates the pupils are too close to the nose of the equipment/lens,while a high positive score indicates the pupils are too close to thetemporal component. Not included in the ECP survey, but it should benoted that it is globally preferably for the pupils to be slightlycloser to the nasal component.

Specific Criteria 3: Provide feedback regarding the horizontal locationof an external component of the eyebrow relative to equipment shape.

Specific Criteria 4: Provide feedback regarding the vertical location ofthe eyebrow relative to the top aspect of the equipment.

Specific Criteria 5: Provide feedback regarding the vertical position ofthe bottom aspect of the equipment relative to the cheeks of the user.

Specific Criteria, 3, 4, and 5 can be evaluated on a similar continuumof negative and positive values.

Specific Criteria 6: Provide feedback regarding the bridge size of theequipment relative to the nose width of the user. A high negative scoreindicates the bridge is too narrow, while a high positive scoreindicates the bridge is too wide.

The results of each ECP survey associated with a dataset of userfeatures, equipment features, and corresponding image of a user wearingequipment may be stored alongside the dataset in an annotated databaseA, discussed in further detail with reference to FIG. 7A. The results ofeach ECP survey, therefore, contribute to a model describing values ofspecific criteria and a model describing values of global criterion,wherein the models are based on the evaluation of the ECP of aspects ofthe proposed combination of the face of the user and the equipment.

In an embodiment, the results of each ECP survey can be used to provideconstraints for mathematical expressions, equality and inequalityconditions, and logical conditions that define each of the specificcriteria model and the global criterion model according to user featuresand equipment features. Coefficients and parameters in such expressionscan be obtained using machine learning-based tools, as will bediscussed.

For instance, such expressions may be based on, for a given set ofspecific criteria (Sci, Sc2, . . . , ScN), user features includingtemporal width of the face of the user, sphenoidal width of the face ofthe user, horizontal positions of gravity centers of eyelid openings ofthe user, mean vertical positions of the eyebrows of the user, length ofthe nose of the user, and the like, and equipment features includingtotal width of the frame of the equipment, dimensions of aspects of theframe of the equipment (e.g., Size A, Size B, Size D, etc.), verticalthickness of the top aspect of the frame of the equipment, horizontalthickness of the frame of the equipment at the level of the hinge, colorof the equipment, material of the equipment, and the like. In oneexpression related to Sc₁, the expression may be written as

Sc₁ =too small if (temporal width—total equipment width

-   -   >20 mm), okay if (temporal width—total equipment width    -   ≤20 mm and temporal width—total equipment width    -   ≥−20 mm), too large else

In one expression related to Sc₂, the expression may be written as

Sc₂ =too small if (sphenoidal width—(2A+D)

-   -   >20 mm), okay if (sphenoidal width—(2A+D)    -   ≤20 mm and sphenoidal width—(2A+D)≥−20 mm),    -   too large else        where A represents ‘Size A’, a dimension of the frame of        equipment, and B represents ‘Size B’, a dimension of the frame        of equipment.

In one expression related to Sc₃, the expression may be written as

Sc₃ =eyebrow too high above the frame of the equipment if(max(Y_EyeBrow)

-   -   −(Y_(top) of the frame of the equipment) >10 mm),    -   eyebrow viewed inside the frames if max(Y_EyeBrow)    -   −(Y_(top) of the frame of the equipment <=−5 mm), okay else

In the above-described examples, thresholds defining boundaries between,for instance, being too small, okay, and too large, can be determined bymachine learning-based approaches. The thresholds may be determined inview of the answer scale in use. For the instance, the answer scale maybe textual bins describing three possible modes or numerical binsdescribing a fit scale between −5 and +5. In an embodiment, thethresholds may be defined by machine learning-based approaches appliedto results of the ECP surveys shown in FIG. 6B. It can also beappreciated that thresholds may be dependent upon relationships andinteractions between specific criteria, wherein Sc₁ may impact Sc₃, forinstance.

Alternatively, and as will be described more with reference to FIG. 8Aand FIG. 8B, the specific criteria model and the global criterion modelmay also be developed according to statistical methods or other machinelearning-based approaches as applied to results of the ECP surveysregarding the specific criteria and global criterion of a given userdataset. Such statistical methods may include linear discriminantanalysis, for instance.

To this end, and according to an exemplary embodiment, an ECP survey maybe directed to a dataset acquired from raw database A 533. The ECPsurvey may include a series of images of the user wearing the equipment502 and, alongside each image, a series of questions regarding specificpoints of suitability of the equipment and the face of the person. Foreach question, the ECP may be given a limited number of possibleanswers. In an example, it may be a scale between −5 and +5, a scalebetween 0 and 10, or a choice of an item in a set of N items. Exemplaryquestions and answers, as submitted to an ECP during completion of anECP survey, are described below.

Question 1. Relative to the width of the face of the user, how do youevaluate the width of the equipment? (a) too small, (b) okay, or (c) toolarge.

Question 2. Relative to the caliber of the equipment, how do youevaluate the pupils of the user? (a) too internal, (b) okay, or (c) tooexternal.

Question 3. What do you think about the position of the external cornersof the eyebrows of the user relative to the caliber of the equipment?(a) too internal, (b) okay, or (c) too external.

Question 4. Relative to the eyebrows of the user, how do you evaluatethe position of the top of the frame of the equipment? (a) too low, (b)okay, or (c) too high.

Question 5. Relative to the cheeks of the user, how do you evaluate theposition of the bottom of the frame of the equipment? (a) too low, (b)okay, or (c) too high.

Question 6. Relative to the nose of the wearer, how do you evaluate thebridge of the frame of the equipment? (a) too narrow, (b) okay, or (c)too large.

The above-described exemplary questions provide an introduction tomyriad features that may be considered during development of thespecific criteria model, as will be shown with respect to FIG. 7Bthrough FIG. 7L. Moreover, though described above with reference to thespecific criteria model, a similar approach may easily be implemented inthe development of a global criteria model.

Accordingly, with reference to FIG. 7A, a flow diagram of generation ofa specific criteria model and a global criterion model will now bedescribed. Specifically, FIG. 7A depicts submission of a user dataset ofannotated database A 736 to a first machine learning approach 704 inorder to generate the specific criteria model 737. In an embodiment,submission of the user dataset of annotated database A 736 to the firstmachine learning approach 704 may also generate the global criterionmodel 747.

Annotated database A 736 includes the user features 711 and equipmentfeatures 721 of raw database A shown in FIG. 6A and the correspondingECP survey results 735 acquired as shown in FIG. 6B. Datasets ofannotated database A 736 can then be provided to the first machinelearning approach 704.

According to an embodiment, the first machine learning approach 704 maybe a linear discriminant analysis or similar approach for determining alinear combination of features that characterizes or separates aplurality of classes of objects or events, including neural networks andthe like.

In an example, wherein the specific criteria model 737 is beinggenerated, the first machine learning approach 704 may be a lineardiscriminant analysis (LDA), wherein the first machine learning approach704 endeavors to explain the ECP survey results 735 by identifyingstatistical laws that link the ECP survey results 735 to user features711 and equipment features 721. For instance, LDA according to the ECPsurvey results 735 may support a law giving the probabilities (p_(a),p_(b), p_(c))_(i)(i=1, 6) that a corresponding user and equipment are ina state a, b, c for a given question, i. The probabilities may berewritten as (p_(a)+p_(b)+p_(c)=1 where 0≤p_(a) ≤1, 0≤1, and 0≤p_(c)≤1),wherein values of (p_(a), p_(b), p_(c))_(i) describe specific criteriaof the model. It can be appreciated that a similar approach may be usedin determining the global criterion model.

Having applied the first machine learning approach 704 to the userdatasets of annotated database A 736, the output of the first machinelearning approach 704 can be appreciated as the specific criteria model737 or as the global criterion model 747, as appropriate.

FIG. 7B through FIG. 7L provide illustrations of representativemeasurements of user features and equipment features that contribute tothe specific criteria model and to the global criterion model.

FIG. 7B is an illustration of a representative measurement of a value oftemporal width of a user, according to an exemplary embodiment of thepresent disclosure.

FIG. 7C is an illustration of a representative measurement of a value ofinternal canthus distance of a user, according to an exemplaryembodiment of the present disclosure.

FIG. 7D is an illustration of a representative measurement of a value ofexternal canthus distance of a user, according to an exemplaryembodiment of the present disclosure.

FIG. 7E is an illustration of a representative measurement of a value ofnose length of a user, according to an exemplary embodiment of thepresent disclosure.

FIG. 7F is an illustration of a representative measurement of a value ofnose length of a user based on a base and a raw as morphologicalfeatures, according to an exemplary embodiment of the presentdisclosure.

FIG. 7G is an illustration of a representative measurement of a value ofmaximum height of the right eyebrow of a user, according to an exemplaryembodiment of the present disclosure.

FIG. 7H is an illustration of a representative measurement of a value ofaverage height of the right eyebrow of a user, according to an exemplaryembodiment of the present disclosure.

FIG. 7I is an illustration of a representative measurement of a value ofa distance between medial corners of eyes of a user, according to anexemplary embodiment of the present disclosure.

FIG. 7J is an illustration of a representative measurement of a value ofa distance between lateral corners of eyes of a user, according to anexemplary embodiment of the present disclosure.

FIG. 7K is an illustration of a representative measurement of a value ofa left pupillary distance and a right pupillary distance of eyes of auser, according to an exemplary embodiment of the present disclosure.

FIG. 7L is an illustration of a representative measurement of a value ofa distance between medial corners of eyes of a user, according to anexemplary embodiment of the present disclosure.

As introduced above, relationships between the ECP survey results, theuser features, and the equipment features may be defined according toimplementation of the first machine learning approach or, in an example,a LDA. Accordingly, when applied to an unknown user dataset includinguser features and equipment features, LDA training will ensure anaccurate classification thereof. Accordingly, FIG. 8A is a graphicalrepresentation of responses to the ECP survey to be used as trainingdata, according to an exemplary embodiment of the present disclosure.

According to an embodiment, FIG. 8A can be appreciated in view of aspecific question from the ECP survey, such as “Relative to the width ofthe face of the user, how do you evaluate the width of the equipment?”Responses to the question may be characterized numerically as (0) toosmall, (1) okay, or (2) too large. FIG. 8A demonstrates responses to thequestion considered as training data. Each response indicated by anumber represents an evaluation of the question as it relates to animage of a user with corresponding equipment. The horizontal axis is thefirst discriminant axis (LD1) and the vertical axis is the seconddiscriminant axis (LD2). Each is a linear combination of the variablesused for the model. In the example of FIG. 8A, the linear combinationscan be written as LD1=0.5291134* Size A+0.4516208* Size D−0.1110664*temporal width−0.1571796* hinge thickness−0.0956928*sphenoidal width andas LD2=0.05257000*

Size A+0.64085819* Size D+0.12367100* temporal width−0 0.09174694* hingethickness−0.15042168*sphenoidal width where ‘Size A’, ‘Size D’,‘temporal width’, ‘hinge thickness’, and ‘sphenoidal width’ are definedas described in FIG. 7B through FIG. 7L.

LD1 and LD2, as shown in FIG. 8A and as defined by the aboveexpressions, provide maximal separation between the answers to thesurvey question: (0) too small, (1) okay, and (3) too large.

Having trained the LDA according to the ECP survey results andidentified expressions of LD1 and LD2 that best separate the responsesthereto, the LDA may be applied to unclassified user features andequipment features. Accordingly, FIG. 8B is a graphical representationof estimated responses to a survey administered to eye careprofessionals, according to an exemplary embodiment of the presentdisclosure. As in FIG. 8B, the LDA provides conditional probabilitiesfor the frame of the equipment to be classified as (0) too small, (1)okay, or (2) too large for a face of a user. The data demonstrated inFIG. 8B reflects a maximal conditional probability.

With reference now to the flow diagram of FIG. 9A, the specific criteriamodel and the global criterion model described above can be applied inthe context of a new database of user datasets in order to generaterelationships between global criterion and specific criteria. An outputof the flow diagram of FIG. 9A can be a decision tree populated byspecific criteria and with each branch of the decision tree ending inglobal criterion such that the global criterion can be explained byvalues of specific criteria that populate a same branch of the decisiontree.

Specifically, user datasets from raw database B 943, having a structuresimilar to that of raw database A and annotated database A, can besubmitted to both a specific criteria model 937 and to a globalcriterion model 947. Outputs of each of the specific criteria model 937and the global criterion model 947 can then be supplied to a secondmachine learning approach 905 in order to generate a decision tree 944.The decision tree 944 may reflect relationships between the outputs ofthe specific criteria model 937 and the outputs of the global criterionmodel 947 as determined by the second machine learning approach 905. Inan embodiment, the second machine learning approach 905 may be aclassification and regression tree. For instance, if there are only afew discrete modes of each specific criteria (e.g., “too small”, “okay”,“too large”), then a classification tree can be implemented. In anotherinstance, if each criteria can be described by a continuum (e.g., “−10to +10), then a regression tree may be implemented.

FIG. 9B is an exemplary decision tree determined according to theoutputs of the specific criteria model and global criterion model and asapplied to the second machine learning approach. It can be appreciatedthat each branch of the decision tree 944 can result in a globalcriterion value 942, wherein the global criterion value 942 is qualifiedby the values of the specific criteria 932 that populate the same branchof the decision tree 944.

Referring now to FIG. 10A, the decision tree generated in FIG. 9A andFIG. 9B can be further processed in order to provide textual contextregarding a fit of equipment relative to a face of a user. To this end,a decision tree 1044 generated by the second machine learning approachof FIG. 9A can be annotated by ECPs, the ECP annotation 1045 therebycontextualizing the decision tree 1044 and generating an annotateddecision tree 1046, as shown in FIG. 10B.

In other words, while prior approaches may provide a global criterionvalue that reside on a scale of one to ten, one being a poor fit and tenbeing a good fit, the present disclosure provides a mechanism by whichthe global criterion value can be qualified according to specificcriteria values generated based on the face of the user and theequipment being evaluated.

In an embodiment, and as in FIG. 10B, a decision tree as in FIG. 9B canbe annotated according to ECP opinion, resulting in an annotateddecision tree 1046. Each branch of the annotated decision tree 1046,therefore, can include a textual context of the global criterion value1042 based on the specific criteria values 1032 that populate the samebranch. For instance, ‘Text2’ 1048, as annotated by an ECP, may be “Thewidth of the frame is a little bit too large for your relatively narrowface”, thereby providing a frame fit evaluation in the context ofspecific features of the frame and the user. In another instance, “Text3” 1049, as annotated by an ECP, may be “The width of this frame suitsyou but your eyes seen through it are a little off-center”. Similarly,such annotation provides a frame fit evaluation alongside contextregarding the position of the features of the user relative to theframe.

In an embodiment, and with reference again to method 200 of FIG. 2 , theannotated decision tree 1046 can provide contextualized evaluations asfit message. The fit messages can then be output to a user to provide aframe fit evaluation, as previously described.

According to an embodiment, the method 200 of the present disclosureallows for a user to understand how and why equipment fits their face.To this end, the user may provide an image of themselves wearingequipment (i.e. eyeglass frames), and user features and equipmentfeatures can be calculated therefrom. The user features and equipmentfeatures may then be applied to the specific criteria model developedabove to determine values of the specific criteria. Application of thespecific criteria values to the annotated decision tree of FIG. 10B mayallow for the determination of global fit criterion, wherein the path ofthe annotated decision tree provides context to the user in the form ofa fit message.

With reference now to FIG. 11 , FIG. 11 is a hardware description of aframe fit evaluation device, according to an exemplary embodiment of thepresent disclosure.

In FIG. 11 , the frame fit evaluation device includes a CPU 1185 whichperforms the processes described above. The frame fit evaluation devicemay be a general-purpose computer or a particular, special-purposemachine. In one embodiment, the frame fit evaluation device becomes aparticular, special-purpose machine when the processor 1185 isprogrammed to perform visual equipment selection (and in particular, anyof the processes discussed with reference to the above disclosure).

Alternatively, or additionally, the CPU 1185 may be implemented on anFPGA, ASIC, PLD or using discrete logic circuits, as one of ordinaryskill in the art would recognize.

Further, CPU 1185 may be implemented as multiple processorscooperatively working in parallel to perform the instructions of theinventive processes described above.

The frame fit evaluation device also includes a network controller 1188,such as an Intel Ethernet PRO network interface card, for interfacingwith network 1199. As can be appreciated, the network 1199 can be apublic network, such as the Internet, or a private network such as anLAN or WAN network, or any combination thereof and can also include PSTNor ISDN sub-networks. The network 1199 can also be wired, such as anEthernet network, or can be wireless such as a cellular networkincluding EDGE, 3G and 4G wireless cellular systems. The wirelessnetwork can also be WiFi, Bluetooth, or any other wireless form ofcommunication that is known.

The frame fit evaluation device further includes a display controller1189, such as a graphics card or graphics adaptor for interfacing withdisplay 1190, such as a monitor. A general purpose I/O interface 1191interfaces with a keyboard and/or mouse 1192 as well as a touch screenpanel 1193 on or separate from display 1190. General purpose I/Ointerface 1191 also connects to a variety of peripherals 1194 includingprinters and scanners. In an embodiment of the present disclosure, theperipherals 1194 may include a 2D or 3D camera, or other image capturedevice configured to acquire images of users, eyewear, and the like.

A sound controller 1195 is also provided in the frame fit evaluationdevice to interface with speakers/microphone 1196 thereby providingsounds and/or music.

The general purpose storage controller 1197 connects the storage mediumdisk 1187 with communication bus 1198, which may be an ISA, EISA, VESA,PCI, or similar, for interconnecting all of the components of the framefit evaluation device. A description of the general features andfunctionality of the display 1190, keyboard and/or mouse 1192, as wellas the display controller 1189, storage controller 1197, networkcontroller 1188, sound controller 1195, and general purpose I/Ointerface 1191 is omitted herein for brevity as these features areknown.

The exemplary circuit elements described in the context of the presentdisclosure may be replaced with other elements and structureddifferently than the examples provided herein. Moreover, circuitryconfigured to perform features described herein may be implemented inmultiple circuit units (e.g., chips), or the features may be combined incircuitry on a single chipset.

The functions and features described herein may also be executed byvarious distributed components of a system. For example, one or moreprocessors may execute these system functions, wherein the processorsare distributed across multiple components communicating in a network.The distributed components may include one or more client and servermachines, which may share processing, in addition to various humaninterface and communication devices (e.g., display monitors, smartphones, tablets, personal digital assistants (PDAs)). The network may bea private network, such as a LAN or WAN, or may be a public network,such as the Internet. Input to the system may be received via directuser input and received remotely either in real-time or as a batchprocess. Additionally, some implementations may be performed on modulesor hardware not identical to those described. Accordingly, otherimplementations are within the scope that may be claimed.

FIG. 12 , is a flow diagram of a method for providing a contextualizedevaluation of an equipment data when other equipment data are provided.When presented with a first set of equipment data where an equipmentdata is missing, the set of data is compared to sets of data of adatabase comprising at least one set of equipment data. The mean valueof the missing data from frames of the database having similar otherfeatures is then calculated. The database then provide suggestions ofequipments having a missing data value equal or close to the calculatedmean value of the missing data and having other values equal or close tothe other values of the first set of equipment data.

Obviously, numerous modifications and variations are possible in lightof the above teachings. It is therefore to be understood that within thescope of the appended claims, the invention may be practiced otherwisethan as specifically described herein.

Embodiments of the present disclosure may also be as set forth in thefollowing parentheticals.

(1) A method for providing contextual evaluation of an eyeglass frame ona face of a user, comprising receiving user data describing features ofthe face of the user, receiving equipment data describing features ofthe eyeglass frame, generating, according to a first model, values for aset of specific criteria describing compatibility between the face ofthe user and the eyeglass frame based on the received user data and thereceived equipment data, the first model trained to associate user dataand equipment data with values of specific criteria, generating, byprocessing circuitry and according to a second model, a value of aglobal criterion based on the generated values for the set of specificcriteria, the second model trained to associate the values of specificcriteria with values of global criteria, determining a messagecharacterizing the eyeglass frame with respect to the face of the user,the message being associated with the generated value of the globalcriterion and with the generated values for the set of specificcriteria, and outputting the message to the user.

(2) The method of (1), wherein the outputting outputs the message to theuser by applying a natural language generator to the determined message.

(3) The method of either (1) or (2), wherein the received user data isbased on an image of the face of the user.

(4) The method of any one of (1) to (3), wherein the received equipmentdata is based on an image of the eyeglass frame.

(5) The method of any one of (1) to (4), wherein the first model isgenerated by applying a first machine learning to a database includinguser data, equipment data, and images of faces of users wearing eyeglassframes, the user data and the equipment data of the database beingassociated with a respective image of the images of faces of userswearing eyeglass frames in the database, the first machine learningbeing trained to associate the user data and the equipment data in thedatabase with reference values of specific criteria and reference valuesof global criteria.

(6) The method of any one of (1) to (5), wherein the reference values ofspecific criteria and the reference values of global criteria aredetermined by human evaluation of the images of faces wearing eyeglassframes in the database.

(7) The method of any one of (1) to (6), wherein the human evaluation isperformed by eye care professionals.

(8) The method of any one of (1) to (7), wherein the first machinelearning is a linear discriminant analysis.

(9) The method of any one of (1) to (8), wherein the second model isgenerated by applying a second machine learning to reference values ofspecific criteria and reference values of global criteria, the secondmachine learning being trained to associate the reference values ofspecific criteria with the reference values of global criteria.

(10) The method of any one of (1) to (9), wherein the second model is adecision tree.

(11) An apparatus for providing contextual evaluation of an eyeglassframe on a face of a user, comprising processing circuitry configured toreceive user data describing features of the face of the user, receiveequipment data describing features of the eyeglass frame, determine,according to a first model, values for a set of specific criteriadescribing compatibility between the face of the user and the eyeglassframe based on the received user data and the received equipment data,the first model trained to associate user data and equipment data withvalues of specific criteria, generate, according to a second model, avalue of a global criterion based on the generated values for the set ofspecific criteria, the second model trained to associate the values ofspecific criteria with values of global criteria, determine a messagecharacterizing the eyeglass frame with respect to the face of the user,the message being associated with the generated value of the globalcriterion and with the generated values for the set of specificcriteria, and output the message to the user.

(12) The apparatus of (11), wherein the first model is generated byapplying a first machine learning to a database including user data,equipment data, and images of faces of users wearing eyeglass frames,the user data and the equipment data of the database being associatedwith a respective image of the images of faces of users wearing eyeglassframes in the database, the first machine learning being trained toassociate the user data and the equipment data in the database withreference values of specific criteria and reference values of globalcriteria.

(13) The apparatus of either (11) or (12), wherein the reference valuesof specific criteria and the reference values of global criteria aredetermined by human evaluation of the images of faces wearing eyeglassframes in the database, the human evaluation being performed by eye careprofessionals.

(14) The apparatus of any one of (11) to (13), wherein the second modelis generated by applying a second machine learning to reference valuesof specific criteria and reference values of global criteria, the secondmachine learning being trained to associate the reference values ofspecific criteria with the reference values of global criteria.

(15) A non-transitory computer-readable storage medium storingcomputer-readable instructions that, when executed by a computer, causethe computer to perform a method for providing contextual evaluation ofan eyeglass frame on a face of a user, comprising receiving user datadescribing features of the face of the user, receiving equipment datadescribing features of the eyeglass frame, generating, according to afirst model, values for a set of specific criteria describingcompatibility between the face of the user and the eyeglass frame basedon the received user data and the received equipment data, the firstmodel trained to associate user data and equipment data with values ofspecific criteria, generating, according to a second model, a value of aglobal criterion based on the generated values for the set of specificcriteria, the second model trained to associate the values of specificcriteria with values of global criteria, determining a messagecharacterizing the eyeglass frame with respect to the face of the user,the message being associated with the generated value of the globalcriterion and with the generated values for the set of specificcriteria, and outputting the message to the user.

Thus, the foregoing discussion discloses and describes merely exemplaryembodiments of the present invention. As will be understood by thoseskilled in the art, the present invention may be embodied in otherspecific forms without departing from the spirit or essentialcharacteristics thereof. Accordingly, the disclosure of the presentinvention is intended to be illustrative, but not limiting of the scopeof the invention, as well as other claims. The disclosure, including anyreadily discernible variants of the teachings herein, defines, in part,the scope of the foregoing claim terminology such that no inventivesubject matter is dedicated to the public.

1. A method for providing contextual evaluation of an eyeglass frame ona face of a user, comprising: receiving user data describing features ofthe face of the user; receiving equipment data describing features ofthe eyeglass frame; generating, according to a first model, values for aset of specific criteria describing compatibility between the face ofthe user and the eyeglass frame based on the received user data and thereceived equipment data, the first model trained to associate user dataand equipment data with values of specific criteria; generating, byprocessing circuitry and according to a second model, a value of aglobal criterion based on the generated values for the set of specificcriteria, the second model trained to associate the values of specificcriteria with values of global criteria; determining a messagecharacterizing the eyeglass frame with respect to the face of the user,the message being associated with the generated value of the globalcriterion and with the generated values for the set of specificcriteria; and outputting the message to the user.
 2. The method of claim1, wherein the outputting outputs the message to the user by applying anatural language generator to the determined message.
 3. The method ofclaim 1, wherein the received user data is based on an image of the faceof the user.
 4. The method of claim 1, wherein the received equipmentdata is based on an image of the eyeglass frame.
 5. The method of claim1, wherein the first model is generated by applying a first machinelearning to a database including user data, equipment data, and imagesof faces of users wearing eyeglass frames, the user data and theequipment data of the database being associated with a respective imageof the images of faces of users wearing eyeglass frames in the database,the first machine learning being trained to associate the user data andthe equipment data in the database with reference values of specificcriteria and reference values of global criteria.
 6. The method of claim5, wherein the reference values of specific criteria and the referencevalues of global criteria are determined by human evaluation of theimages of faces wearing eyeglass frames in the database.
 7. The methodof claim 6, wherein the human evaluation is performed by eye careprofessionals.
 8. The method of claim 5, wherein the first machinelearning is a linear discriminant analysis.
 9. The method of claim 1,wherein the second model is generated by applying a second machinelearning to reference values of specific criteria and reference valuesof global criteria, the second machine learning being trained toassociate the reference values of specific criteria with the referencevalues of global criteria.
 10. The method of claim 6, wherein the secondmodel is a decision tree.
 11. An apparatus for providing contextualevaluation of an eyeglass frame on a face of a user, comprising:processing circuitry configured to receive user data describing featuresof the face of the user, receive equipment data describing features ofthe eyeglass frame, determine, according to a first model, values for aset of specific criteria describing compatibility between the face ofthe user and the eyeglass frame based on the received user data and thereceived equipment data, the first model trained to associate user dataand equipment data with values of specific criteria, generate, accordingto a second model, a value of a global criterion based on the generatedvalues for the set of specific criteria, the second model trained toassociate the values of specific criteria with values of globalcriteria, determine a message characterizing the eyeglass frame withrespect to the face of the user, the message being associated with thegenerated value of the global criterion and with the generated valuesfor the set of specific criteria, and output the message to the user.12. The apparatus of claim 11, wherein the first model is generated byapplying a first machine learning to a database including user data,equipment data, and images of faces of users wearing eyeglass frames,the user data and the equipment data of the database being associatedwith a respective image of the images of faces of users wearing eyeglassframes in the database, the first machine learning being trained toassociate the user data and the equipment data in the database withreference values of specific criteria and reference values of globalcriteria.
 13. The apparatus of claim 12, wherein the reference values ofspecific criteria and the reference values of global criteria aredetermined by human evaluation of the images of faces wearing eyeglassframes in the database, the human evaluation being performed by eye careprofessionals.
 14. The apparatus of claim 11, wherein the second modelis generated by applying a second machine learning to reference valuesof specific criteria and reference values of global criteria, the secondmachine learning being trained to associate the reference values ofspecific criteria with the reference values of global criteria.
 15. Anon-transitory computer-readable storage medium storingcomputer-readable instructions that, when executed by a computer, causethe computer to perform a method for providing contextual evaluation ofan eyeglass frame on a face of a user, comprising: receiving user datadescribing features of the face of the user; receiving equipment datadescribing features of the eyeglass frame; generating, according to afirst model, values for a set of specific criteria describingcompatibility between the face of the user and the eyeglass frame basedon the received user data and the received equipment data, the firstmodel trained to associate user data and equipment data with values ofspecific criteria; generating, according to a second model, a value of aglobal criterion based on the generated values for the set of specificcriteria, the second model trained to associate the values of specificcriteria with values of global criteria; determining a messagecharacterizing the eyeglass frame with respect to the face of the user,the message being associated with the generated value of the globalcriterion and with the generated values for the set of specificcriteria; and outputting the message to the user.